Abstract 


This brief is based on the report 
Next Generation Accountability: A 
Vision for School Improvement Under 
ESSA, which offers alternatives to 
single accountability indicators 
that are inadequate for supporting 
the dual goals of deeper learning 
and college- and career-ready 
graduates. With the impending 
implementation of the Every 
Student Succeeds Act, states are 
about to gain considerably more 
authority and autonomy over the 
design of school accountability 
systems. Consequently, there is 

an opportunity to design systems 
that produce information that 
genuinely explains how schools 
and school systems are meeting 
the learning and developmental 
needs of all students. With this 

in mind, the report offers a set of 
design principles and a conceptual 
framework for next generation 
accountability, provides an example 
of an Educational Quality and 
Improvement Profile, and makes 
recommendations for potential 
next steps. 


The full report is available at 
https://learningpolicyinstitute. 
org/product/next-generation- 
accountability. 
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Next Generation Accountability: 
A Vision for School Improvement 
Under ESSA 


Introduction 


In his book The Best Practice,t Charles Kenney tells a story about quality 
improvement in health care that resembles the narrow use of accountability 
indicators in education. Kenney describes how the medical director of Allegheny 
Hospital, Rick Shannon, was called before a regional task force on quality 

and safety. Shannon was to defend the hospital’s poor cardiothoracic surgery 
outcomes, which he did with a reasoned and articulate argument as to why the 
indicators misrepresented actual performance. At the conclusion of his explanation, 
the chair of the task force told Shannon that he had missed the point of the meeting. 
The intent was not to defend the hospital’s performance—rather, the chair exhorted, 
“We’re here for you to tell us what you have learned from this and how we can all 
share in the learning so that we don’t ever have to have poor results.” 


A similar story unfolds every year in states and communities across the country 
when school accountability indicators are released to the public. High-ranking 
schools are lauded and praised, whereas those with poor scores become 

the target of public scorn and are subjected to a flood of discordant, external 
interventions marketed as effective solutions to their problems. In reality, we find 
ourselves in a situation similar to what Kenney describes in the health care case— 
we lack profound knowledge to explain why and how some schools achieve the 
outcomes they do. It seems that learning, as an objective of school accountability, 
has been lost in the frenzy of ranking schools with simplistic metrics. 


A new accountability approach is needed, one in which learning replaces 
ranking as the driver of quality improvement. With the Every Student Succeeds 
Act (ESSA), states have an opportunity to design and develop accountability 
systems that support all schools in their improvement journeys. We use this 
brief to chart a path toward next generation accountability by (a) advancing a set 
of design principles and a conceptual framework for new models, (b) providing 

a sample profile of educational quality and improvement, and (c) suggesting 
recommendations for moving forward. 


Next Generation Accountability 


Next generation accountability seeks to move beyond schooling “in which no 
child is left behind” to a system in which “children are healthy and thriving.” 
This is a policy sea change—a focal shift from compliance with external 
mandates to strengthening schools and empowering the pursuit of standards 
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for educational excellence. It shifts the task from test score gains to fostering an environment characterized by deeper 
learning, a condition foundational to the goal of college and career readiness.” The framework casts a broad, inclusive 

net in recognizing those who share responsibility for building a state’s capacity to achieve these lofty goals. It replaces 

a summative judgment of school performance, rendered as a single indicator, with multiple summative and formative 
indicators delivered as comprehensive school profiles. Adaptive interventions replace one-size-fits-all approaches, with the 
intent of assuring a high-quality, stable faculty for every school. Table 1 summarizes differences between first generation 
and next generation accountability frameworks. 


Table 1 
Contrasting First and Next Generation Accountability 
First Generation Accountability Framework Next Generation Accountability Framework 
No Child Left Behind All children healthy and thriving 
Focus on improving test scores Focus on fostering deeper learning 
Compliance to meet proficiency standards Capacity building 
Schools accountable to state Shared accountability 
Summative emphasis Formative emphasis 
Single composite indicator of student performance Multiple indicators of whole system performance 
One-size-fits-all interventions Adaptive interventions 
Replace teachers and leaders Retain and support quality teachers and leaders 


Source: Adapted from Adams, C.M., Forsyth, P.B., Ford, T.G., Ware, J.K., Barnes, L.B., Knojasteh, J., et al. (2015). Next gen- 
eration school accountability: A report commissioned by the Oklahoma State Department of Education. Oklahoma City, OK: 
Oklahoma Center for Education Policy (The University of Oklahoma) and the Center for Educational Research and Evaluation 
(Oklahoma State University). Retrieved from http://okedpolicy.org/wp-content/uploads/2015/12/Next-Generation-School- 
Accountability-Report-Final.pdf. 


Design Principles 

College, career, and citizenship readiness is a significantly more challenging vision, setting our schools on a path well 
beyond test score proficiency. This new vision emphasizes the cognitive and noncognitive competencies expected for 
success in a postindustrial society and economy. What kind of accountability framework will facilitate progress toward 
this newly identified and specified vision? Three principles derived from the education policy and accountability literature 
provide guidance. 


Principle One: Shared Accountability—Responsibility for school success is distributed. 


The goal of “shared accountability” is to create an accountability environment in which all participants recognize 

their obligations and commitments in relation to each other.? In contrast with past accountability models, a shared 
accountability framework is designed to render a comprehensive account of how each part of the educational system, and 
the system as a whole, is performing relative to the vision of college and career readiness. As Linda Darling-Hammond and 
colleagues urge, “each level of the system should be held accountable for the contributions it must make to produce an 
effective system.”* 


The principle of shared accountability reminds us that in a complex enterprise such as public education, performance 
responsibility is distributed across the system’s components, not contained in any one group of stakeholders. Whereas 
previous accountability frameworks held schools alone responsible for student test scores, shared accountability 
recognizes a more fundamental set of student outcomes and identifies the critical contributions of the “whole village” to 
school effectiveness. 
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Principle Two: Adaptive Improvement—Take schools where they are and move them forward. 


First generation accountability assumed that districts and schools are alike in their capacity to become effective. Next 
generation accountability acknowledges that school capacities differ greatly and that a one-dimensional continuum derived 
from student test results does not effectively describe school quality and capacity, explain performance, or identify targets 
for improvement. Embracing the concept of adaptive improvement necessitates a system that is flexible and responsive to 
particular school conditions, fitting interventions to specific challenges and needs. 


Adaptive improvement acknowledges that schools are in different places on their paths to effectiveness and that 
without essential resources and appropriate processes in place, schools will be unable to achieve even modest goals. 

A school lacking stable, quality leadership and teaching staff, for example, is at a different improvement stage than one 
whose teachers are experienced and have worked together successfully over time. It follows that schools have different 
information and resource needs; moreover, their abilities to respond to and benefit from an accountability framework are 
different. Thus, a state’s approach to working with schools should be contingent on current assessed conditions at each 
school and an individual school’s particular need for support, expert assistance, and other resources. 


Principle Three: Informational Significance—Put relevant and useful information in front of responsible actors. 


Next generation accountability systems seek to collect and distribute significant information salient to the work and 
interests of all stakeholders. Stakeholders are likely to regard a broad palette of functionally significant indicators replacing 
a single composite indicator as informational rather than controlling, thereby motivating them to act.° 


The information system designed to support next generation accountability should recognize the dual needs—compliance 
with federal mandates and the particular improvement needs of a state’s schools. It should also address the different 
information needs of the state, district, school, and parents. Information about resources, processes, and a variety of 
outcomes can enable sense making by legislators who allocate school resources, by local school boards that develop 
school policy, by principals and teachers who design and adapt the instructional environment to fit local conditions, and by 
parents who want to make good choices for their children. 


Conceptual Framework 

Next generation accountability follows a systems model of school function (see Figure 1). The systems model depicts 
outcomes as a consequence of the quality of school resources together with the quality of school processes. Resource 
and process elements are defined as a set of critical capacities that ultimately focus instruction on the knowledge, 
skills, and dispositions that ready students for the workforce or advanced learning. When school outcomes are deemed 
unsatisfactory, the model and its constituent capacities enable stakeholders to identify, explain, and target needed 
change. Thus, while deeper learning and college and career readiness are ultimate goals, they also provide direction for 
capacity building throughout the system. 


Organizational capacity is defined as school resources that are foundational to the development and maintenance of 
high-quality teaching and learning. 


Home and community capacity is composed of the social and material support that surrounds children, and describes 
relative opportunity that varies across families and communities. 


Instructional capacity characterizes the health of school processes. It is defined as the ability and readiness of the 
school’s teaching corps to design and deliver appropriate, challenging, and goal-related instruction to all students. 


Learning capacity also characterizes the health of school processes. It is defined as the ability and readiness of the 
school’s students to engage in the work of mastering the knowledge, skills, and dispositions necessary for college, career, 
and citizenship readiness. 


Deeper learning and college- and career-readiness reflect desired outcomes of the educational system. We consider 
two aspects of deeper learning. The first is the opportunities provided by the school for students to transfer knowledge 
and skills developed in one setting to new situations, contexts, and problems through a set of cognitive and intra- and 
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Figure 1 
Key Concepts in Next Generation Accountability 


SCHOOL RESOURCES SCHOOL PROCESSES SCHOOL OUTCOMES 


Organizational Instructional Capacity Deeper Learning 
Capacity 


Home and Community Learning Capacity College- and Career- 
Capacity Ready Graduates 


DEEPER LEARNING FOCUS 


Source: Adapted from Adams, C.M., Forsyth, P.B., Ford, T.G., Ware, J.K., Barnes, L.B., Khojasteh, J., et al. (2015). Next 
generation school accountability: A report commissioned by the Oklahoma State Department of Education. Oklahoma City, 
OK: Oklahoma Center for Education Policy (The University of Oklahoma) and the Center for Educational Research and 
Evaluation (Oklahoma State University). Retrieved from http://okedpolicy.org/wp-content/uploads/2015/12/Next-Genera- 
tion-School-Accountability-Report-Final.pdf. 


interpersonal competencies.® The second is the deeper learning outcomes required for graduates to enter a career, 
extended training, or specialized education without the need for remediation. 


An Educational Quality and Improvement Profile 

Consistent with the design principles and conceptual framework behind next generation accountability, we have 
developed and propose for consideration an Educational Quality and Improvement Profile. EQuIP is envisioned as both 
an accountability and school improvement tool. It provides accountability indicators of student performance, as well as 
resource and process indicators that are related to school outcomes. On the accountability side, EQuIP establishes a 
level of transparency in reporting summative student outcomes far exceeding a composite summative index. It presents 
accurate and clear accountability information on deeper learning and college and career readiness, tracks changes in 
achievement gaps, and reports student growth. On the improvement side, it points to formative conditions and resources 
in schools that need to change to improve observed outcomes. 


EQulPs benefit every school. Consistently low-performing schools desperately need nuanced data to inform deliberate 
strategies and focused support for improvement. Even the highest-performing schools cannot afford to be static, since 
each group of learners has quite different needs. Thus, EQuIP is envisioned as a comprehensive profile with multiple uses, 
depending on the specific needs of each school community (see Figure 2). All stakeholders can find useful information 

in EQuIPs. Parents seeking their children’s greatest well-being, policymakers seeking efficiency and effectiveness, and 
educators seeking to create and sustain thriving schools will find accurate and useful information about different aspects 
of the life and health of schools. 


EQuIPs are committed to the following measurement guidelines: 


1. Resource, process, and outcome indicators should report changes over time. 

2. Indicators should be assessed with appropriate frequency and minimal disruption. 

3. Credible scientific evidence substantiating the validity and reliability of measures should be clearly reported. 
4. Resource and process indicators should measure conditions, attitudes, structures, and behaviors. 
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5. Outcome indicators should report achievement differences by student subgroups. 

6. Outcome indicators should enable the identification of focus schools and priority schools, and should reward schools 
consistent with criteria set for federal waiver requirements. 

7. Indicators and measurement methods should change over time in response to the continuous evaluation of the school 
accountability framework. 

8. Indicators should be reported and used to avoid gaming practices and distortion of school performance. 


Recommendations for the Design and Implementation of Next Generation 
Accountability 


In this final section, we translate next generation accountability into a set of recommendations for state and local 
education agencies. These recommendations have three distinct targets: (a) accountability policy; (b) alignment of 
standards, assessments, and accountability; and (c) school, district, and state capacity building in support of the 
accountability framework. 


Accountability policy 
1. Do not use a single, summative index to report accountability information. Outcome evidence should clearly report 
student performance toward deeper learning and college- and career-ready standards, changes in student performance 
over time, and achievement gaps. 
¢ Single summative indices cannot be used to make valid and reliable judgments of school quality. 
¢ Single summative indices do not provide useful information for improvement. 
¢ Outstanding and equitable outcomes should be the goal of every school. 
¢ Variation in student outcomes needs to be studied and understood so targeted action can address performance gaps. 
¢ Trend data provide a more accurate account of student and school performance compared to the instability of time 
point estimates. 
¢ Evidence on achievement equity and performance trends allows for more reliable identification of schools in need of 
state intervention. 


2. Multiple indicators of capacity for quality improvement should be part of a school profile. 
¢ Knowledge formation includes understanding what, how, and why improvement is or is not happening. 
¢ States and districts can better identify schools in need of intervention by understanding capacity differences among 
schools. 
¢ We must learn the lessons of Campbell’s Law. At no time should social measures be used in high-stakes or otherwise 
summative decision making—to do so completely undermines their validity as measures of social processes.’ 


Curriculum standards, assessments, and accountability 
1. Develop a new, coordinated system of multiple assessments, both formative and summative, to measure student 
learning using the operational definitions we have constructed for deeper learning and college and career readiness. 
Such a system should be defined by the following major features: 
¢ Higher-order cognitive skills are assessed. 
¢ Critical skills are assessed with high fidelity. 
¢« Assessments are benchmarked to international standards. 
e Assessments are instructionally sensitive and educationally valuable. 
* Assessments are valid, reliable, and fair.® 


2. In addition to measures of student learning, indicators of dispositional and behavioral constructs associated with 
deeper learning and college and career readiness should be included. 
3. The system should emphasize frequent use of formative assessments, particularly those embedded in instruction. 
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Figure 2 


An Example of an EQuIP Page Reporting Cognitive Outcome Indicators 
for an Elementary/Middle School 


DEEPER LEARNING: COGNITIVE 


SAMPLE ELEMENTARY SCHOOL 


DEEPER 
LEARNING 


STUDENT PERFORMANCE 
Bar graphs indicate the 
distribution of student 
achievement by proficiency status: 
advanced, proficient, limited 
knowledge, and unsatisfactory. 
The AMO target represents the 
goal of reducing by 50% the 
number of students scoring below 
proficiency by 2020. 
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CHANGES OVER TIME 

Pie graphs indicate the change in 
reading and math proficiency 
status for a cohort of students 
from 3rd grade in 2014 to 5th 
grade in 2016. 
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ACHIEVEMENT GAPS 

Line graphs report changes in 
reading and math achievement 
gaps for student subgroups with 
30 or more students. 


Deeper learning refers to the 
expectation that schools develop 
in students cognitive and 
noncognitive competencies 
needed for effective participation 
in the workforce and active 
citizenship. Initially, proficiency 
rates, changes in proficiency 
rates over time, and 
achievement gaps serve as 
indicators of the cognitive 
component of deeper learning. 
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Source: Adapted from Adams, C.M., Forsyth, P.B., Ford, T.G., Ware, J.K., Barnes, L.B., Khojasteh, J., et al. (2015). Next 
generation school accountability: A report commissioned by the Oklahoma State Department of Education. Oklahoma City, 
OK: Oklahoma Center for Education Policy (The University of Oklahoma) and the Center for Educational Research and 
Evaluation (Oklahoma State University). Retrieved from http://okedpolicy.org/wp-content/uploads/2015/12/Next-Genera- 
tion-School-Accountability-Report-Final.pdf. 
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4. Consideration should be given to grade-span testing of achievement outcomes. It may not be necessary or even 
desirable to test every student in every subject every year. 
5. Assessment results should be reported by student subgroups to highlight performance gaps. 


School, district, and state capacity building 

1. Asystem-wide culture grounded in “learning to improve.” Change under next-generation accountability is dynamic 
and context specific. A new support system needs to communicate its importance for enhancing local as well as state- 
wide educational improvement. For genuine change to take root, it is essential that the purposes, processes, and goals 
of improvement be shared within and across sites. The ideas of science of improvement methodology and Networked 
Improvement Communities proposed by the Carnegie Foundation for the Advancement of Teaching and Learning are 
two research-based frameworks that seem to have particular utility for accomplishing this.° 


2. Development of strong pedagogical data literacy skills. In this framework, data are meant to enhance decision 
making, not to be a substitute for it. Pedagogical data literacy*® refers to technical ability related to one or more of the 
following areas: numeracy and statistical knowledge, facile use of data analysis software, and what might be considered 
general inquiry skills, such as the ability to formulate and test appropriate questions and to develop solutions based on 
findings.** Research demonstrates that the demands we place on school professionals to select and use appropriate 
data for decision making have far outstripped our attention to the need for better training on how to go about this 
process in a meaningful way.*? A next generation accountability system would maintain focus on data-driven decision 
making, but would also ask school professionals to become expert on the use of data to explain outcomes and decide 
on appropriate interventions. This recommendation has strong implications for teacher- and leader-preparation 
programs. Preservice teachers and aspiring leaders need the knowledge and skill set to develop meaningful classroom 
assessments, to interpret assessment results, and to make meaning from student performance information. 


3. Prioritization of resources for sustaining ongoing improvement. System-wide availability of resources such as time, 
access to expertise, and collaborative opportunities are critical.t? Meaningful learning occurs in collaboration with others, 
and having easy access to colleagues, instructional coaches, and other leaders, as well as outside experts, will ensure 
that school staff can see a wide range of possibilities in addressing issues of teaching, leading, and learning. Allocating 
time and improving access to expertise and collaborative opportunities will likely require increased school funding or 
some reallocation thereof. The intentional allocation of these key resources sends a signal about the importance of such 
endeavors and creates conditions in which schools and school personnel can achieve critical learning. 


4. Acoherent structure of state-level support for learning to improve, including a strong Longitudinal Data System 
infrastructure. Learning to improve has to be part of a larger, coherent framework of state-led support. The alignment 
of a strong culture around learning to improve and the allocation of key resources to support change need to be present, 
focused, and coordinated with next generation accountability. This structure could build on existing support resources in 
states, such as school support offices, but it will require significant expansion to accommodate disparate learning needs 
across states and districts. The current resources provided to most of these support networks are insufficient. States 
and their stakeholders are urged to partner with intermediate service agencies (including state and local universities) 
where capacity already exists to genuinely assist in efforts to develop a next generation accountability system. 


5. Educator labor market policy that supports the above elements. Little progress in the preceding elements will be 
made without addressing key educator labor market challenges, and this includes a reexamination of current policy tied 
to the supply of experienced educators in particular states and regions of the United States. Furthermore, our proposed 
system of supports will require more professional educators who have had considerable past experience working in 
schools. In some cases, this may require an examination of teacher retirement incentive laws (i.e., “double-dipping” 
regulations) to determine if they preclude former teachers from participating in these new support positions. If so, then 
providing exemptions so that highly qualified former educators can participate in these positions will be necessary. 
These policy changes are not without precedent: Some states, such as Michigan, have responded to labor market 
shortages by relaxing these regulations. 
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Conclusion 


This vision for Next Generation Accountability, unlike current test-based accountability, draws on the best social science 
evidence we have to date on how individuals and organizations are motivated, as well as how they learn, grow, and thrive. 
Although this is no guarantee of success, we believe this is a more fruitful starting point for school improvement policy and 
practice than one based on conjecture or ideas about motivation that are not supported by evidence. 


The increased authority, autonomy, and flexibility given to states under ESSA represent a welcome opportunity for states to 
get improvement right. What states and local education agencies need now is a plan to leverage their new-found freedom 
into success in moving their educational systems forward. 


The work is substantial. This report contains the broad strokes of a plan for states and other local education agencies to 
use in making schools work for all children, families, and communities. In the true spirit of improvement, however, this 
vision necessarily leaves the finer details about how to execute it up to local policy actors. 
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